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INTRODUCTION 



This indexing study was part of a more general study of the 
relationship of the Bibliography of Agriculture (B of A) to fif- 
teen other secondary services. Reports were prepared earlier on 
the coverage and overlap studies of this effort.* 

The objective of this indexing study was to determine the re- 
lationship of B of A indexing to the indexing performed by 15 other 
services with agricultural R&D interests. The study was done by a 
review of data about these services, as well as a comparative study 
of the indexing done by each of the services for a common body of 
citations . 

The National Agricultural Library staff specified the following 
(secondary services to be studied with regard to their relationship 
to B of A: 

Biological Abstracts (BA) 

Biological & Agricultural Index (BAI) 

Chemical Abstracts (CA) 

Dairy Science Abstracts (DSA) 

Engineering Index (El) 

Forestry Abstracts (FA) 

Horticultural Abstracts (HA) 

Index Medicus (IM) 

Index Veterinarius (IV) 

Meteorological & Geoastrophysical Abstracts (MGA) 

Pesticides Documentation Bulletin (PDB) 

Public Affairs Information Service (PAIS) 

Textile Technology Digest (TTD) 

Tobacco Abstracts (TA) 

World Agricultural Economics & Rural Sociology Abstracts 
(WAERSA) 

All of the above services, with the exception of BAI, IM, PDB, and 
PAIS, provide abstracts in addition to a citation. This group of 
services was intended to represent a cross-section of services that had 
a relationship to agriculture. These services are represented through- 
out this report by the abbreviations noted above. 



♦Bourne, Charles P. Characteristics of Coverage by the Bibliography ox 
Agriculture of the Literature Relating to Agricultural Research and 
Development . Report No. IGC-PA-69-38. 1 June 1969. 

Bourne, Charles P. Overlapping Coverage of Bibliography of Agriculture 
by 15 Other Secondary Services . Report No. IGC-PA-69-39. 1 June 1969. 
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II. MAJOR CHARACTERISTICS OF THE SERVICES STUDIED 
A. GENERAL 

Some of Lhe general characteristics (e.g., publisher, estimated 
circulation^ of cue services studied are listed in summary form in 
Table I. All of the services are English-language, and all are pub- 
lished in the United States or England. Several of the services 
such as IV and WAERSA are more directly related to agriculture than 
others such as CA and BA. Published reports of the operating prac- 
tices and procedures of several of these services are cited in the 
bibliography at the end of this section. For this report, particular 
attention was given to finding reports that discussed the indexing 
practices of those services. 

In indexing, a useful distinction can be made between access 
points which can be extracted from the data elements in the citation 
and those which are added from examination of the full publications 
cited. Access points such as author, corporate author, patent or 
report number, when present in the citation can be pulled out easily 
for indexing. In general, subject indexes, including corporate names 
as subjects and geographic location indexes must be prepared from the 
full text. Keyword-in-title indexes are an exception in that they 
provide some subject access using only the citation. It is also true 
that author and corporate author indexes compiled from the citations 
may serve sometimes as subject approaches. However, as a general 
principle it is subject access which is not implicit in the trans- 
cription of the citation, and which requires intellectual as contrasted 
with clerical effort. Therefore in considering the indexing provided 
by the various services the subject and non-subject features have been 
distinguished. 

Figures 1 and 2 show full size, typical entries in the printed 
author and subject indexes, respectively, of these services. This 
presentation shows the great variety in size and style of typography 
used by the various services. Many use all capitals, which is a 
consequence of computer production but is more difficult to read. In 
addition, this kind of printout is often photo-reduced, making it even 
less legible. There is also variation in the content of the indexes, 
e.g. , brief citation, full citation, or reference number only. The 
full page format of each service could not be shown here, but this too 
would indicate further visual variety. More specific characteristics 
of the indexing practices of each of the services are described in the 
following sections. 
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Fig. 1 (concluded) 

Typical Examples of Author Indexes of Each Service 
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Fig. 2 (continued) 

Typical Examples of Subject Indexes of Each Service 
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B. DATA ELEMENTS USED IN THE PRINTED CITATION 

There is wide variation in the numbers and combinations of 
elements of data that are included in the printed citations of 
each service. Direct comparison is difficult because of differing 
coverage; for example, a service which covers only English language 
material has no option regarding inclusion of a translated title. 

Thirty-eight elements were identified, of which B of A uses 
25. Other services ute from 13 to 32, with t^* % average number being 
23. Inclusion of first and second authors, title in full, volume 
number, inclusive paging, and year are the elements common to 
all services. Data elements used by more than half of the other 
services but not by B of A include corporate author location, cross- 
reference to related citation, and price. A third of the services, 
but not B of A, also include author affiliation and article title in 
original foreign language. 

Use of the data elements, as determined by examination of 
printed citations, can be seen in Table II. 

The data elements and characteristics of the bibliographic 
records in corresponding computer files of bibliographic data for 
several of these services have been ifentified and described separately 
in a report by the Auerbach Corporation. 

C. INDEXING AND ACCESS FEATURES PUBLISHED IN EACH ISSUE 

1 . N on-Subject Access 

The non-subject access features pjrovided in each printed issue 
by each of the services are summarized in Table III. Examples of author 
indexes for most of these services were given earlier in Fig. 1. Almost 
all services provided an author index with each issue, but few provided 
an author affiliation or address. More than half of the services pro- 
vided access by corporate author, generally by including corporate 
authors in the author index. Organization names as subjects were given 
in subject indexes by some services. Only the two NAL publications pro- 
vided separate organization indexes. Through 1967, 3 of A provided an 
Organization Index which included corporate names as authors or as sub- 
jects. Beginning in 1968, a separate corporate author index was pro- 
vided, and organizations as subjects were included in the subject index. 
Beginning in late 1967, PDB provided a separate organization index 
covering corporate authors and sponsoring organizations. 

None of the services provided a report number index, and only one 
service provided a patent number index. Nine of the services provided 
some indication of how copies of the cited materials could be obtained, 
usually by an introductory note announcing that copies could be ob- 
tained from a general source. 
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2 . Subject Access 

The basic styles and forirats of subject indexes of these 
services were illustrated earlier in Fig. 2. Details of the sub- 
ject access features provided with each issue of each of the ser- 
vices are given in Table IV. Several of the services include 
cross-reference statements in the index. A few of the services 
have a subject index separate from the citations, and directly 
yield only a reference number, as shown in the samples from B of A, 
BA Biosystematic and CROSS, CA, MGA, and WAERSA in Fig. 2; and 
two services have indexes of the same general type that directly 
yield a brief citation instead of a reference number (see the 
samples from BA B.A.S.I.C. and PDB in Fig. 2). Seven of the 
services simply list the full citation under a single subject 
access point, in the manner of a classified bibliography. 

D. INDEXING AND ACCESS FEATURES PUBLISHED ANNUALLY OR CUMULATIVELY 

1 . Non-Subject Access 



Two of the services (IV, PDB) provide no annual indexes or 
cumulations.* One service (CA) cumulates semi-annually and quinquennially 
only. All but four of the services provide annual author indexes, with 
generally less than the full name, and never with any indication of the 
author 1 s affiliation or address. Those services which provide corpor- 
ate author indexing in their monthly issues, a majority, provide such 
indexing annually. None of these services provide an annual report 
number index, and only one of them provides a patent number index. 

Several other special features are provided in cumulations, notably 
the cumulative Translations Index of MGA and Bibliography of Medical 
Reviews of IM. Fig. 3 includes examples of these special indexes. De- 
tails for the non-subject access features for each of the services are 
given in Table V. 

2 . Subject Access 



Some of the services cumulate the subject indexes from single 
issues; others provide only an annual index. Only one service (CA) 
regularly cumulates subject indexes over a period longer than annual. 

One other service (HA) has published a cumulated index for 1931-1960. 

Most of the services which have an annual subject index post a 
reference number under one or more subject access points. Four services 



* Since the completion of this study, NAL has announced the publication 
of the first semi-annual Cumulative Index to PDB, to cover the 
January 1 to June 7, 1968 issues. 
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Table V 

Indexing and Access Features (Non-Subject) Provided Annually and Cumulatively by Selected Secondary Servi 



publish the complete citation in the cumulations. Almost air. of 
the services included cross-reference statements. Some of the 
special indexes included in annual indexes and cumulations are 
special subject indexes, notably the following: 

Formula Index (CA) 

Index of Ring Systems (CA) 

Hetero-Atom-in-Context Index (CA) 

Species Index (FA) 

Geographic indexes (FA, MGA, WAERSA) 

Examples of some of these are included in Fig. 3. Details of 
the subject access features provided annually and cumulatively for 
each of the services are given in Table VI. 

E. AN EXAMPLE OF THE INDEXING GIVEN TO THE SAME DOCUMENT BY SEVEN 
SERVICES 

One specific example of the variation in format and extent of 
indexing treatment given by the various services is given by the 
citation shown in Fig. 4. This illustration shows the indexing 
provided by seven different services for the same journal article. 
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Semiannual cumulative index announced after the completion of this study 
Cumulated from subject indexes in each issue 

Table VI 

Subject Access Features Provided Annually and Cumulatively by Selected Secondary Services 



B of A 
Citation 
Monthly Issue 
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PLANT HOST VIRUSES 
(See also "Virus Diseases" under Phytopathology 

t 112737. BOSCH, L., E. M. BONNET-SMITS, an.: J. VAN DU IN. (Dep. 
Biochem., State Univ. Leiden, Ieiden, Noth.) In -Uu bre akage of ; , i.r- 
ntp yJljow mosaic vir u s RNA and in situ ag k regr>. to ci 1 ne fragmi. 
VIROLOGY 31(3): 453-460. Ulus. 1967. -Turnv v >w masaif virus 
(TYMV) has been treated at alkaline pK (10.5-11.. ,ir.u h.gh ior..c 
strength (l.Olf JKC1] at 5Q°for 8 minutes. Accordur.; to Kaper and Hal- 
perin such a treatment causes in situ brea*:ag£ a: :;ie viral RNA cnsir., 
yielding fragments of uniform siie (about 5 $,. b. present p-pc. 
is demonstrated that in situ fragmentation m accompanied by ir. &;tu 
aggregation of the RNA fragments. The aggregate can be released as 
such from the capsid with phenol and sedimer.;s more rapidly and 
more uniformly than TYMV-RNA. It Is assumed tnat each aggregate 
molecule is derived from one virus particle and has adopted a struc- 
ture which is more compact than that of TYa 1V*RNA. Deaggregation, 
which Is essentially irreversible outside of the capsid, can be accom- 
plished by (a) heating at 55* (or 2 minutes; (b) treatment with dimethyl- 
su If oxide; (c) tne successive removal of dlva'ent and monovalent ca- 
tions. Below limiting temperatures aggregates of intermediate sizes 
persist wtvm the heating Is prolonged. Possible models lor the struc- 
ture of the aggregate are discussed. — Authors. 
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BOSCH, L. ; BONNET-SMITS, E M.; VAN DUIN, J. 

In situ breakage of turnip yellow mosaic virus RNA and 
in situ aggregation of the fragments. Virology 31(3): 
453-60, Mar. 1967. graphs. 

Turnip yellow mosaic virus (TYMV) has been treated at 
alkaline pH (10. 5-11. 0) and high ionic strength (1. 0 M KC1) at 
30° for 8 minutes. According to Kaper and Halperin (1965) 
such a treatment causes in situ breakage of the viral RNA 
chain, yielding fragments of uniform size (about 5 S). In the 
present paper it is demonstrated that in situ fragmentation is 
accompanied by in situ aggregation of the RNA fragments. 

The aggregate can be released as such from the capsid with 
phenol and sediments more rapidly and more uniformly than 
TYMV-RNA. It is assumed that each aggregate molecule is 
derived from one virus particle and has adopted a structure 
which is more compact than that of TYMV-RNA. Deaggrega- 
tion, which is essentially irreversible outside of the capsid, 
can be accomplished by heating at 55° for 2 minutes; treatment 
with dime thylsu If oxide; the successive removal of divalent 
and monovalent cations* Below limiting temperatures aggre- 
gates of intermediate sizes persist when the heating is pro- 
longed. Possible models for the structure of the aggregate are 
discussed. (Abstract) 1106 
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III. RELATIVE INDEXING 
A . PROCEDURE 

This part of the study concentrated on an examination of the 
indexing actually applied by these services to a number of selected 
citations. The basic materials for this part of the study consisted 
of more than 2,000 citations from B of A that were indexed both by 
B of A and one or more of the fifteen other secondary services. 

These original citations were from a uniform random sample of 
over 5,000 citations that were extracted from the 1967 B of A and 
used for the previously mentioned study of the overlap of B of A and 
these other services.* This uniform random sample of over 5000 
citations (5.4 percent of 1967 B of A publication) was selected from 
the total 1967 B of A production of 94,238 citations. Because the 
B of A citations are printed only once in the subject section of an 
issue, and are assigned a unique serial number as indicated in a 
representative page shown in Fig. 5, it was easy to obtain a uniform 
random sample. Every 20th citation was selected for analysis, as 
well as every 326th citation, to boost the total sample size to over 
5,00C, This sample of citations formed the basis for many of the 
findings of the reports on this project. 

Each citation in the samplewas annotated to show such things as 
its national origin, language, and form of publication. Different 
parts of the citation (e.g., author, title, date) were also annotated 
with data field tags for subsequent computer processing and file 
building. The citation was keypunched in a format for the INFOL file 
management system, for subsequent running on a CDC-3800 computer.** 

A total of 170 data elements (i.e., data fields) were established for 
each of the citation records in this file. Further description of this 
computer file and the associated procedures are given in earlier IGC 
project reports.*** These reports describe the searching which iden- 
tified the citations indexed jointly by B of A and another service. 



* Bourne, Charles P. Overlapping Coverage of Bibliography of Agriculture 
by 15 Other Secondary Services . Report No. IGC-PA-69-39 . 1 June 1969. 

** 3600/3800 Computer Systems INFOL Reference Manual. Control Data 
Corporation, Palo Alto, California, July 1966. 

i 

*** Bourne, Chiles P. op .cit . 
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MANI, E., and RUSTERHOLZ, F. Culture of 
apple seedlings for rearing apple insects. (Gel 
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Oct. 15, 1966. 80 Sch9 

MARSHALL, J. An apple grower charts course. 
Mountaineer Grower 262:24-28. Mar. 1966. 

80 M86 

MAURER, K. J. New apple varietv: Mutu-as 
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MOCHALOVA, T. I. Climatic conditions ana 
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Includes discussion. 
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70286 



70287 



70288 

70289 



70290 



70291 



70292 



70293 



Fig. 5 

160 

Representative Page of 1967 B of A 



44 



All of the index terms obtained from the services for these 
citations were added to the computer file with the associated 
citations. This computer file of citations and the accompanying 
indexing data from each of the services were used to generate 
several types of printouts that provided the starting materials 
for this part of the study. 

The index terms for these sample citations were obtained as 
a result of cooperative arrangements with several of the services. 

The comprehensive nature of this study would not have been possible 
without their generous support and assistance, particularly Chemical 
Abstracts Service, BioSciences Information Service of Biological 
Abstracts, and the National Library of Medicine. The literature 
searching done for our earlier overlap study identified those 
B of A citations in the sample that were covered by these 15 other 
services. Special computer runs were made to extract a bibliography 
of B of A citations known to be covered by each of the other services 
(e.g., CA) and to print them in a sequence convenient for the other 
service (e.g., by CA abstract number). This bibliography was sent 
to that service for their searching and annotation necessary to fur- 
nish the associated index terms. In most cases, the terms were ex- 
tracted from original worksheets, computer files, and other records 
of the service. In a few cases, the services re-indexed the material 
in a way that v/as representative of their original production index- 
ing. A summary of the procedures used to obtain the index terms for 
each service is given in Table VII. It is felt that even though some 
special procedures and re-indexing were required, the index terms used 
for this study generally represent the depth and quality of indexing 
regularly given by these services. 
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Table VII (Concluded) 



B. AVERAGE NUMBER OF SUBJECT ACCESS POINTS PER CITATION 



1. Absolute Number 



Measures of the average number of subject access points 
per citation for BA* and CA** have been published. IGC staff 
analyzed several other services to get additional measures. These 
data are shown below. 



Service 


Total Number 
of Citations 
in Sample 


Total Number of 
Subject Access 
Points in Sample 


Average Number 
of Subject 
Access Points 
Per Citation 


Data Analyzed 


B of A 


94,238 


231,898 


2.5 




NAL computer 
listing of all 
index terms 
assigned to B of A 
citations in 1967 


BA 


5,209 


B.A.S.I.C. 44,160 
Bicsystem- 
atic 6,600 

CROSS 35,813 

Total 86,573 


8.5 

1.3 

6.9 

16.6 




October 15, 1967 
issue* 


CA 


93,000 


580,700 


6.2 




January-June 1964 
(Vol . 60)** 


MGA 


754 


1,394 


1.8 




December 1967 
issue 


PDB 


842 


3,333 


4.0 




May 10, 1968 
issue 


TA 


2,966 


4,119 


1.4 




All 1967 issues 


TTE 


10,926 


15,490 


1.4 




1967 annual 
cumulation 


These measures are not 


directly comparable 


because 


the data 


analyzed were 



not selected according to any consistent criteria, and thus they have no 
direct interrelation. There is no intended correlation by subject, time 
period, or quantity. 



*Zabriskie, K.H. , Jr. and A. Farren, "The B.A.S.I.C. Index to Biological 
Abstracts/’ American Journal of Pharmaceutical Education , v. 32, no. 8, 
May 1968, p. 190 




**Wetsel , F.R., "Time Studies in Producing Subject Indexes for Chemical 
Abstracts , " Journal of Chemical Documentation , v. 5, no. 4, November 1965, 

p. 212. 
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2. Relative Number 



A more meaningful measure may be obtained by comparing the 
number of subject access points provided by B of A and another service 
for the same citations. The computer file of citations covered joint- 
ly by B of A and each of the other services included the index terms 
supplied by the services as described earlier. By using the INFOL 
file manipulation capabilities, a special printout was made which 
listed and counted all the index terms supplied by a particular ser- 
vice in regular issues and in computer data banks to those citations 
indexed jointly with B of A (see Fig. 6). Another listing was made 
of the terms supplied by B of A and each other service to the citations 
•covered by both (see Fig. 7). From these printouts, a count of the 
terms supplied by both services for the identical collections of 
citations (i.e., citations that hcjve index terms recorded from both 
services) was made. This count provided a relative measure of the 
average number of subject access points provided by each service 
for exactly the same collection of original documents. These data 
are illustrated in Fig. 8 and summarized in Table VIII . 

These data show that two services (BA and TTD) furnished more 
than four times as many access points as B of A for the same citations. 
These two services provided an average of more than 10 subject access 
points per citation. However, in the case of TTD this number includes 
the large number in the computer data bank; the printed issues of TTD 
averaged only 1.67 terms per citation. The data indicate that for the 
same material four printed services provided at least 30 percent more 
index terms, and at least three other services provide roughly as many. 
Including the terms in computer files, six of the eleven services thus 
studied provided more index terms than B of A. 

C. RELATIONSHIP OF TERMS APPLIED BY WO SERVICES TO THE SAME DOCUMENT 
1 . Procedure 



a. The Need for a Measure 



One of the main objectives of this indexing study effort was to 
shew the relationship of B of A indexing to the indexing provided to the 
same material by other secondary services. Comparisons have been made 
between the language of indexing systems,* but the examination that we 



* Painter, Ann F. An Analysis of Duplication and Consistency of Subject 
Indexing Involved in Report Handling at the Office of Technical Services. 
Ph.D. Thesis, Rutgers, The State University, 1963. 
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Fig. 7b 

Examples of Computer Printouts 
of Index Terms Contributed by 
B of A and Other Services 
to the Same Document 
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Examples of Computer Printouts 
of Index Terms Contributed by 
B oi A and Other Services 
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Fig. 8 

ative Number of Subject Access Points Per Citation Provided by B of A 
and 13 Other Services for the Same Citations 
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Table VIII 

Relative Number of Subject Access Points Per Citation Provided by B of A 
and 13 Other Secondary Services for the Same Citations 



felt would be the most revealing was a comparison, on an article- 
by-article basis, of the terms that were actually applied to the 
same document by B of A and the other services. To our knowledge 
this had not been done to any extent in prior studies, although 
one recent article reviewed the indexing coverage given to four 
B of A citations by several services, and made some general state- 
ments in conclusion,* and another study listed the index headings 
in B of A and other services under which a group of citations were 
found.** It was our hope that a methodology could be developed by 
which some absolute and objective measures could be established 
to describe the degree and type of relationship between assigned 
indexing terms. To our knowledge, such measures or procedures had 
not been developed or suggested prior to this study. 

Considerable effort was spent in planning and testing such a 
methodology to be used to describe the relationship of terms assigned 
by multiple services to the same document. Particular emphasis was 
placed on developing some normalized numeric measure that would: (1) 

have meaning independent of the sample size used or the particular 
services studied; (2) be repeatable by other experimenters; and (3) 
permit direct comparison of several services. Initial efforts were 
concentrated on developing such a procedure for a two-service 
situation, with the understanding that the procedure could be general- 
ized to more than two indexing services if desired. Variation in tech- 
nique was necessary to cover the wide difference between single- and 
multiple-term indexing. The developed measure is believed to be satis- 
factory, and yields results indicative of the relationship of the various 
indexing services studied. However, the difficulty in making objective 
and consistent judgments about subject terms is not to be underestimated. 
While consistency can be checked to some extent by duplication of efforts 
by several workers, objectivity cannot be assured without further studies 
of how the particular indexes are used. 

b. The Relationship Matrix and Measure 

The procedure finally adopted for this study involved the use 
of a relationship matrix which displayed, for a given citation, the 
subject index terms applied to it by B of A and one other service. 
Examples of such matrix worksheets and the index terms provided by 
several services are shown in Figs. 9a-h. Using the indexing infor- 
mation arranged on such worksheets, a research analyst worked on one 
citation at a tim to: (1) compare each term of one service against 

each of the other terms assigned by the second service; (2) make a 
judgment of the degree of association (e.g., terms not judged to be 



* Bystrom, Marcia. "Agricultural Information: Can You Find It With the 
Index?" Special Libraries , v. 59, no. 9, November 1968, pp. 712-717. 

**Martyn, J. and M. Slater. "Tests on Abstract Journals," Journal of 
Document at i on , v« 22, no. 4, December 1964, pp. 212-235; v. 23, no. 1, 
March 1967, pp. 45-70 
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alphabetically identical) for each of the term pairs; and (3) annotate 
the matrix for the corresponding term-pair. A list of possible types 
of meaningful term pair associations was developed and used for this 
purpose in order to ensure rigor and consistency in the judgments and 
identification of term associations. Much of the judging was done by 
research analysts who had a general knowledge of the subjects. Another 
research analyst, a librarian who was even more familiar with classi- 
fication and indexing techniques* reviewed all of the judgments and 
made some additional ones. 

The types of term-pair relationships and associations that were 
defined and used in this study depended somewhat on the type of indexing 
performed by the particular service being studied. For example, some 
types of relationships were possible with hierarchical systems that had 
no counterparts in coordinate or keyword indexing systems and vice versa. 
For this reason, two separate lists of possible relationships were es- 
tablished for use by the research analysts: one for services with single 

term index entries, and one for services with multiple term or hier- 
archical index entries. The first list (for indexes with single-term 
entries) was used on comparing BA B.A.S.I.C. and Biosystemat ic indexes, 
DSA, PDB, and TTD computer index terms with B of A. (As printed, the 
BA Biosystemat ic index and DSA f s subject index contain multiple term 
entries, but only single terms vere supplied by the services to IGC, ) 

The second list was used in compari; % BA CROSS index, BAI , CA, El section 
headings, HA, IM MEDLARS, MGA, PAIS, TA, TTD printed index, and WAEKSA 
index terms with B of A. 

After analysis using these two separate lists, a combined or com- 
posite list was created, as shown in Table IX, to show all possible re- 
lationships noted in the analysis of both types of index terms. 

The variations allowed in the determination of alphabetical identity 
are listed in Table X. 

In the transcription of hierarchical index terms, commas, hyphens, 
and slashes were all used by the various services to denote different 
hierarchical levels. These punctuation marks were also ignored in this 
analysis . 

c . Sample Siz e 

At least twenty documents with joint indexing (i.e., documents 
indexed by both B of A and one of the other services) were studied for 
each service unless that number of documents was unavailable. The 
specific documents in the sample of twenty were selected by a uniform 
sampling of the available population of jointly indexed documents 
(e.g., every 15th IM citation of the jointly-indexed population was 
selected) . It was felt that this sample size would provide a good test 
of the procedure, as well as a relatively stable estimate of the measure 
of relationship. A total of 305 instances of documents with joint 
indexing are still available in the data base for further study in this 
manner by IGC at a later date. 
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1 . 



Same alpha entry, with allowable variations 



Whole term identical, with allowable variations (la II) 

B of A more general: other service has more terms, or 
more specific terms following alpha entry (12) 

B of A mare specific; B of A has more terms, or more 
specific terms following alpha entry (lb 13} 

• Equal level of indexing 

: Terms following alpha entry are synonymous (14) 

: Terms following alpha entry are not synonymous (15) 

. Relationship of terms following alpha entry is not 
ascribable to above categories (16) 

II. Different alpha entry; concept of alpha entry related to a 
concept in terms of contracted service 

. B of A more general; other service more specific (Ilf 112) 

• B of A more specific; other service more general (Hi 113) 

: Single word or term of other service is identical 

with a term following the B of A alpha entry term (Ila) 

: Single word or term of other service is synonymous with 

a term following the B cf A' Alpha entry term (lie) 

: Single word or term of other service is synonymous with 

first word of B of A term; B of A has succeeding term(s) lid) 

• Equal level of indexing 

: Terms are synonymous (lib I 14) 

: Terms are inverted (III) 

: Terms are not synonymous (I 15) 

Table IX 

Classes of Relationships Noted in Comparing Index Terms Assigned by Two 

Services to the Same Document 



Relationship of terms is not ascribable to above categories (lie IJ6) 



: Single word or term of other service is a subset of a 

term following' the B of A alpha entry term (II g) 

: Single word or term of other service is a subset of the 

first word of B of A term; B of A has succeeding term(s) 

III. No relation between two particular terms 

IV Relation undetermined because outside subject competence of analyst 



Table IX (Concluded) 

Classes of Relationships Noted in Comparing Index Terms Assigned by Two 

Services tc the Same Document 



0 



/jERiC 

* 



(Hh) 



V 



Variant spellings of the same word 
gray vs grey 
color vs colour 
litchee vs lychee 

Word plurals 

nut vs nuts 

Word form 

package vs packaging 
vibrio vs vibrionic 
arachnids vs arachnida 

Punctuation and special characters 
sub-tropical vs subtropical 
leaf, diseases vs leaf diseases 

Articles 

vines and nuts vs vines, nuts 
fleas in fruit vs fleas, fruit 

Abbreviations 

lawns, etc., diseases vs lawns, diseases 
per^skiopsis vs pereskiopsis spp. 

US vs United States 



Table X 

Variations Allowed in Determining Identical Index Terms Assigned by 
Two Services to the Same Document 
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d. 



Source Data 



The original working documents used to provide the source 
data for this part of the study were the series of computer print- 
outs of the citations jointly indexed by B of A and the other ser- 
vices. A few examples of such printouts were shown in Fig. 7a-d. 

2 . Finding s 

a . Services With Single Term index Entries 



The detailed analysis of the index terms applied by two 
services to the same document, using the relationships listed earlier 
in Table IX, yielded the data illustrated in Figs. lOa-c, and sum- 
marized in Table XI. These data are given in terms of numbers and 
percentages of the possible term-pair relationships (i.e*, each term 
of B of A compared with each term supplied by the other service for 
tie same document) that were of each of the four types described in 
Table IX. These data lead to the following specific observations for 
the comparison of B of A with the BA B.A.S.I.C. and Biosystematic 
indexes, the indexes for DSA and PDB, and the TTD computer index: 

Depending on the service, 4 to 16 percent of the 
term-pairs studied were entered under alphabetically 
identical terms, i.e., the first word of B of A f s 
term was alphabetically identical to the single term 
used fcy the other service, assuming the allowable 
variations of the types noted in Table X. An average 
of 10 percent of all the term-pairs analyzed for each 
of the services had the same alphabetic entry in th" s 
way. An average of only one half of the one percent 
of all the term-pairs were completely identical, i.e., 

B of A used an identical single term index entry, 
with no terms following the entry term. (See Fig. 10a 
and Type I relationships in Table XI) . 

From 28 to 56 percent of the term-pairs studied were of 
the same or related concepts . An average of 46 percent 
of all of the term-pairs analyzed for each of these ser- 
vices were of the same or related concepts. (See Fig. 10b) 

From 43 to 71 percent of the term-pairs studied were un- 
related in concept . An average of 52 percent of all of 
the term-pairs analyzed for each of these services were 
unrelated in concept* (See Fig. 10c) 

b. Services With Multiple or Hierarchical Term Index Entries 



The term-pairs for these services were analyzed in the same man- 
ner as done for the single-term services described in the previous section. 
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Fig. 10a 

Index Term Relationships Observed in Citations Indexed Jointly 
by B of A and Other Services That Use Single Term Index Entries 
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Fig. 10b 

Index Ten Relationships Observed in Citations Indexed Jointly 
by B of A and Other Services That Use Single Term Index Entries 
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B.A.S.I.C. (20) Biosystematic (20) DSA (20) PDB (70) TTD (20) Combined Data 
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The numbers in parentheses refer to the number of citations sampled from the file of jointly covered citations and used in this analysis. 



The resulting data are illustrated in Figs, lla-c, and summarized 
in Table XII. These data led to the following specific observatiors 
for the comparison of B of A with the indexes for BAI , CA, El, HA, 
MGA, PAIS, TA, the BA CROSS index, the TTD printed index, WAERSA’s 
old and new indexing systems; and IM MEDLARS* terms: 

Depending on the service, 0 to 37 percent of the 
term-pairs studied were entered under alphabetically 
identical terms, i.e., at least the entry words of both 
terms in a pair were alphabetically identical, assum- 
ing the allowable variations of the types noted in 
Table X. An average of 13 percent of all the term- 
pairs analyzed for these services had the same 
alphabetic entry. An average of only one half of 
one percent of all the term-pairs were completely 
identical, i.e., not only the entry word but also 
all the terms following it were alphabetically 
identical. (See Fig. 11a and Type I relationships 
in Table XII) 

From 44 to 98 percent of the term-pairs studied were 
of the same or related concepts . An average of 66 per- 
cent of all of the term-pairs analyzed for each of these 
services were of the same or related concepts. (See 
Fig. lib) 

From 2 to 54 percent of the term-pairs studied were 
unrelated in concept . An average of 30 percent of 
all of the ''erm-pairs analyzed for each of the ser- 
vices were unrelated in concept. (See Fig, 11c) 

c . Combined Data 

A composite summary of the term-pair data for all of the ser- 
vices is given in Fig* 12, 



* In this section, ”lM MEDLARS” refers to the total number of terms 
supplied by NLM, with no distinction as to which terms appeared in 
the printed IM, 
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Pig. 11a 

Index Term Relationships Observed in Citations Indexed Jointly by B of t 
and Other Services That Use Hierarchical or Multiple Term Index Entries 
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Other Services That Use Hierarchical or Multiple Term Index Entries 



WAERSA WAERSA Printed BA IM 

New * . 01d * Index El PAIS ** CA + HA TA + BAI + CROSS MGA MEDLARS 

(20) (20) (20)* (29) 06) (20) (25) (20) (20) (20)* (5)** (20)* 
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Fig. 11c 

Index Term Relationships Observed in Citations Indexed Jointly by B of A 
and Other Services That Use Hierarchical or Multiple Term Index Entries 
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Other Services That Use Hierarchical or Multiple Terra Index Entries 
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Index Term Relationships Observed in Citations Indexed Jointly by B of A and 
Other Services That Use Hierarchical or Multiple Term Index Entries 



TTD WAERSA WAERSA 

Printed Index TA New System Old System Combined Data 

( 20 )* ( 20 ) * ( 20 )* ( 20 )* 
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3. 



Estimate of Measurement Error 



As mentioned earlier, the comparison on an article-by-article 
basis of all possible combinations of index terms supplied by two 
services required a considerable amount of time. For that reason, 
the sample size for this analysis was kept relatively small, generally 
20 citations if this number was available. However, in order to de- 
termine how sensitive the estimates of term relationship type per- 
centages were to the sample size used, several estimates were made 
from increasingly larger sample sizes. PDB was the single-term service 
studied in this way. BAI , IM MEDLARS, and WAERSA were the multiple- 
term services studied in this way. 

The estimates of percentage of types of term-pair relationships 
that were obtained with increasing sample sizes for single-term and 
multiple-term services are given in Tables XDI and XIV respectively. 

In general, this data indicates that an absolute variation of about 
+ 5 percent might be added to the percentage values determined 
earlier for each of the types of term-pair relationships, because 
of the smaller samples sizes actually used in this analysis.* This 
would not appear to signif icantly change the findings obtained with 
these samples, and reported in the previous section. 

4 . Unique Subject Access Points 



A separate analysis was made of the index terms applied to 
the same sample of citations. In this analysis, each term was ex- 
amined with respect to all of the terms supplied by the other service 
for that citation, instead of looking simply at pairs of terms. A 
judgment was then made as to whether a particular entry term was dif- 
ferent in concept from all terms of the other service . A further 
distinction was made as to whether or not terms following the entry 
bore a relation to terms of the contrasted service. This analysis 
yielded a number representing the unique subject access points pro- 
vided for a citation by B of A and a number representing the unique 
access points provided for the same citation by the other service. 
These numbers were totalled for B of A and for each other service. 

The totals were then combined to get a grand total of unique subject 
access points provided for that set of citations, together with per- 
centages representing the contribution of B of A and of each of the 
other services to this grand total. These data are illustrated in 
Fig. 13 and given in detail in Table XV. 

Of the services which, like B of A, provided multiple term in- 
dexing, CA provided the largest percent of unique subject access 



* This would probably be a smaller figure, say +2 percent, for the 
estimate of the percent of term* pair relationships that have the 
same alphabetic entry. 
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Effect of Sample Size on Determination of Value of Index Term Relationships Observed in Citations Indexed Joinciy uy 
B of A and Another Service (PDB) That Uses Single Term Index Entries 
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Effect of Sample Size on Determination of Value of Index Term Relationships Observed 
in Citations Indexed Jointly by B of A and Other Services That Use Hierarchical or Multiple Term Index Entries 
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Citations Indexed Jointly by B of A and Other Services That Use Hierarchical or Multiple Term Index Entries 



PERCENT OF TOTAL UNIQUE SUBJECT ACCESS 
POINTS PROVIDED BY OTHER SERVICE 
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Multiple Term Indexes, Single Subject Access Point Studied 



El 

WAERSA New 


15 


85 


15 


85 


WAERSA Old 




15 


85 


TA 


23 


77 





60 



81 



84 



88 



50 


50 


50 


50 



40 



in 



12 
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Incomplete data since only one access point was studied for each of these services. 
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Fig. 13 

Percent of Total Unique Subject Access Points Provided 
by B of A and 13 Other Services 



points (88.2 percent). Of these a large number were chemical 
compounds. Three services of this type provided a smaller per- 
centage of unique terms than B of A: El (25 percent), WAERSA 

(15 percent) and TA (22.7 percent). Five other services covered 
a range from 84 percent to 50 percent. Two services, BAI and 
PAIS, also provided multiple term indexing, but should be con- 
sidered separately, since only the one entry under which a searcher 
found the citation was available. 

Of the services which use single term entries, the largest 
percent of unique access points was provided by the TTD Computer 
Index (96 percent), and the least by BA Biosystematic (31.6 percent). 
The three others (J3A, B.A.S.I.C., PDB, and DSA) varied from 82 
percent to 45 percent. 



Q ERiC 



82 



IV. RELATIONSHIP OF B OF A SUBJECT LIST TO TERMS APPLIED BY OTHER 

SERVICES 

In addition to comparing the index terms that were actually 
applied jointly to individual documents by B of A and other services, 
a study was also made of the extent to which all of the terms used by 
the other services on the jointly covered documents were also included 
in the B of A list of subject headings, as represented by the B of A 
1967 annual subject index (see sample page in Fig. 14). The object- 
ive of this effort was to obtain another, and independent, measure of 
the degree of association between the indexing of B of A and other 
services. This measure represents the degree of potential match, and 
is less sensitive to the frequency of use of a term by each of the 
services. The working data for this part of the study consisted of 
an alphabetically arranged computer listing of the index terms used 
by each of the other services for indexing the sample documents 
covered by B of A. A sample page from such a computer listing was 
shown in Fig. 6. 

As with the study of comparative indexing that was done on a 
citation-by-citation basis, a number of types of relationships could 
be defined for the match between the other services 1 terms and the 
terms in the B of A authority list. However, because of the large 
number of possible matches in the authority list, it would have been 
too difficult to apply exactly the same set of relationships. For 
this reason, this analysis effort was restricted to determining only 
the number of Class I relationships described earlier (i.e., same 
alphabetic entry, with allowable variations) that existed between 
the terms assigned by the other services and the terms on the B of A 
list . 



This analysis was done by checking to see whether each of the 
other service T s terms had a Class I relationship to a term in the 
B of A annual index. If it had such a relationship, then a note was 
made to indicate: (l) whether the whole term, as used by ^he other 

service, appeared in B of A; or (2) whether just the entry word was 
alphabetically the same in both B of A and the other service; or (3) 
whether B of A had only a see reference under that entry term. The 
variations allowed in assigning alphabetic identity were the sane as 
those used in comparing index terms assigned by two services to the 
same document (see Table X) 

Data for this comparison are illustrated in Fig. 15 and summarized 
in Table XVI, anu show that a large percentage (48 to 93 percent) of 
the index terms actually used by the other services on the sample 
B of A citations are already included in the B of A subject authority 
list. 
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CANADA 
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Fig . 

Sample Page from 1967 B of A 
Subject Index 
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•♦REFERS TO PUBLICATION q F u.S.O.A.. 

state agriculture experiment station 

or EXTENSION SERVICE. 



A! 1NOEXEO FROM ABSTRAf, JOURNAL • 

full text gf original MAY nct at 
available. 
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Percent of Total Number of Different Terms Supplied 
by This Service That Had the Same Alphabetic 
Entry Term as Those in the B of A Subject List 



Fig. 15 

Percent of Other Services’ Index Terms in B of A Subject Index 
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Relationships Observed in a Comparison of the 1967 B of A Subject Index 
with the Index Terms Supplied by the Other Services 



In contrast to the findings described in earlier sections, 
i.e., that a relatively f.mall percent of terms assigned ic the same 
citations by B of A and one other service had the same alphabetical 
entry with allowable variations, this analysis revealed a much 
higher percentage of terms with the same alphabetical entry. This 
indicates that there is a high degree of overlap in the vocabulary 
used by B of A and other services, even though the choice of terms 
for a given citation may differ considerably,. 

It should be noted that this count was based on alphabetical 
relationships rather than subject relationships or application of 
a term to a particular article. The same word or term might be 
common to both B of A and another service but have an entirely 
different meaning in each index. For example, the terms "slashers" 
and "slashing" used by TTD were judged to have a Class I relation- 
ship with the term "slash* as used by B of A. However, B of A's 
use of "slash" was as a forestry term, and the articles indexed 
by B of A under this term had no relation to 'slash" used by TTD 
to index textile processing. This factor would work to discount 
somewhat the degree of actual commonality of the word lists. 
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V. RELATIONSHIP OF B OF A TFHMS TO TITLE WORDS 

Several prior studies have investigated the extent to which 
terms taken from the document title are related to terms that 
were or would have been supplied by manual indexing, with or 
without an indexing authority list.* Several of the services 
participating in this study (PDB, BA) made extensive use of 
title word indexing. It was of interest to this study to de- 
termine the relationship of present B of A indexing to possible 
title word indexing. 

A. PROCEDURE 

It was assumed that title words were incorporated or re- 
flected to some extent in the B of A indexing. In order to de- 
termine the extent of this inclusion, two uniform random samples 
of one hundred citations each were taken from a computer listing, 
in citation number order, of all index terms applied by N\L to 
the citations appearing in the 1967 B of A (see Fig. 16) . The 
first sample of 100 consisted of every 1000th citation from this 
serial numbered list, beginning with citation number 1000, plus 
every 15,000th citation, beginning with citation number 15,001. 
The second sample of 100 consisted of every 1000th citation be- 
ginning with citation number 500, plus every 15,000th citation 
beginning with citation number 15,501. 

The number of unique words in the set of B of . A index terms 
for each citation and the number of these B of A index terms 
which also appeared in the title of each citation were counted. 
These results are summarized in Table XVII, broken into sample 
groups of fifty in order to show the consistency of results with 
increasing sample size. 



* J. 0 T Connor, "Correlation of Indexing Headings and Title Words 
in Three Medical Indexing Systems . M American Documentation , 
v. 15, no. 2, April 1964, p. 96. 

J. P. Knable II, "An Experiment Comparing Keywords Found in 
Indexes and Abstracts Prepared by Humans with Those in Titles." 
American Documentation , v. 16, no. 2, April 1965, pp. 123-124. 
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B. FINDINGS 

The results given in Table XVII indicate that on the average 
42 percent of the index term words applied by B of A to a citation 
also appeared in the titles of the documents being indexed. This 
suggests that: (1) the titles are descriptive of the subject con- 

tent of the documents, and (2) the indexing was influenced by 
title words. 
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VI. RELATIONSHIP OF B OF A SECTION HEADINGS TO THOSE USED BY 

OTHER SERVICES 

The primary section headings used by B of A, though printed 
in alphabetical order, are similar to a classification system. 

They are general subject categories which cover the whole field 
of agriculture, regardless of the number of articles which fall 
within any one category. In B of A an article is listed once, 
under a general heading, and references are made to it from the 
applicable specific index te^ms in the subject index. Although 
the primary headings are subdivided, they are never very specific. 
Throughout B of A no heading is more specific than a type of crop 
or genus of animal or insect. 

The rationale for general headings is that the reader approach- 
ing via the subject headings is interested in a general approach 
to a general subject. Grouping articles under the general headings 
should allow the reader to use B of A as a current awareness tool 
as well as a reference work. To find very specific information, 
he would use trie index. 

It would be useful to the user if, after scanning though the 

B of A sections, he could also scan the other abstracting and in- 

dexing services in a similar manner in order to add to the complete- 
ness of coverage of a general subject, preferably by looking under 
the same subject heading as that used in B of A. Unfortunately, 
a look at the section headings used by other services for the same 

B of A citation shows that the reader generally cannot do this. 

The subject headings used by the 15 other services studied here 
are generally unlike those used by B of A or by each other. This 
observation is based upon an examination of the printed section 
headings used by those services for the B of A citations that they 
covered. This observation was facilitated by a summary printout 
of section headings for the overlapped citations, as shown in 
Fig . 17. 



The differences in the headings are not due solely to the 
simple substitution of synonyms, but rather to differing degrees of 
specificity in indexing and the different points of view of the in- 
dexes. Furthermore, each of the services is different enough so 
that learning the vocabulary of one does not help much in predicting 
what to look under in another. The following citation should serve 
as an introductory example: 
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Fig. 17 

Sample Page of Summary 
Printout of Printed Section 
Headings Used by 01 her Services 
for B of A Citations 



B of A Citation Number: 



86820 



Title: Hormonal control of reproduction in 

lactating and non- 1 act ating anestrous 
ewes , 



B of A heading: Animal industry - shec^ and 

goats - breeds and breeding 



BA heading: 

BAI heading: 
CA heading: 



Pharmacology - general - 
reproductive system 

Lactation 

Hormones 



DSA heading: Physiology - general 

As a first set of specific and illustrative examples, Table XVIII 
lists the section headings used by three other services (BA, CA, DSA) 
for eight citations from the B of A section heading: Agricultural 

Products - Dairy Products - Analysis and Composition. DSA, which is 
specifically concerned with this field, put the same eight citations 
under six different headings. CA and BA employ broad headings in 
the same manner as B of A, but CA used four different headings for 
seven of these citations, and BA used three different headings for 
six of these citations. 



A second set of examples is given in Table XIX, in which seven 
citations from various B of A sections on veterinary medicine are 
given with the corresponding section headings from four other services 
(BA, BAI, CA, DSA). A third set of examples is given by the three 
unrelated citations in Table XX. It should be noted that BAI probably 
included most of these citations under other section headings in 
addition to those shown in Tables XVIII-XX; however, only one heading 
was picked up in this study. 

With all of the services examined, one point was clear: almost 

the entire contents of any of the other services would have to be 
scanned in order to find a significant fraction of the subject coverage 
of any group of citations under a given B of A section heading. 
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Section Headings Used by B of A and Three Other Services for the Same Citations in the Field of Agricultural Products 



B of A Citation 

Number and Title B of A Section Heading BA S^»ion Heading BAX Section Heading CA Section Heading PSA Section Heading 
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Section Headings Used by B of A and Four Other Services for the Same Citations in the Field of Veterinary Medicine 
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VII . SUMMARY 



1. There is wide variation in the numbers and combinations of 
data elements used in the printed citations of each service. Of 
38 identifiable data elements found in printed citations in the 
services studied, B of A employed 25, compared to an average of 
23 for all the services. Six data elements were found to be 
common to all. Data elements used by more than half of the other 
services but not by B of A included corporate author location, 
cross reference to related citation, and price. 

2. Format and typography varied considerably among all of the 
printed indexes. Author indexes were fairly consistent in giving 
author surname ar.J. initials, and citation number. Subject indexes 
were also fairly consistent in yielding only a citation number. 

3. A majority of the services provided an annual subject index 
and/or an annual cumulation of the citations, although specific 
patterns of cumulation varied. Only two services had no annual 
indexes or other cumulations. 

4. B of A was the only service providing a separate organization 
index. Other services provided other types of specialty indexes, 
such as the geographic indexes of FA, MGA, and WAERSA, and other 
special features such as review articles. 

5. For the same source material, six of eleven comparable services 
provided substantially more subject access points per citation than 

B of A did; two services provided more than four times as many subject 
access points as B of A. 

6. When comparing the index terms actually provided by each of the 
other services and by B of A for the same citations, an average (for 
single term, and for multiple or hierarchical term services, res- 
pectively) of: 

10-13 percent of term pairs had the same alphabetic 
entries 

46-66 percent of term pairs were the same or related 
concepts 

52-30 percent of term pairs were unrelated in concept. 

Less than one percent of the term-pairs were entirely alphabetically 
identical . 

7. Where there were differences in indexing information applied to the 
same citation, the other services, rather than B of A, generally pro- 
vided the additional or unique subject access points. 
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8. A large percentage (48 to 93 percent, depending upon the service) 
of the index terms actually used by the other services on the sample 

B of A citations had the same alphabetic entry as a word in the B of A 
subject index for 1967. This is in contrast to the relatively small 
percent of term-pairs with the same alphabetic entry, as identified in 
the article-by-article comparison. 

9. On an article-by-article basis, approximately 42 percent of the 
index term words assigned by B of A to an article coincide with words 
in the title. 

10. On an article-by-article basis, there is very little agreement or 
commonality in the printed section headings used by these services for 
the same citations. 

In addition to the above substantive findings, this study effort 
was also successful in developing and extending the measurement and 
evaluation techniques applicable to this type of examination. A con- 
siderable amount of useful source data was also compiled in a computer 
data base in a form suitable for further detailed investigation. 

The following generalizations with regard to B of A can be made 
on the basis of the specific findings noted above: 

In thinking of merging or matching the B of A terms 
with the index terms contributed by other services, 

NAL should recognize the difficulty of making such 
matches simply on the basis of an alphabetic match 
of lists of items, as might be done with a computer 
program. Even if all of the exception routines 
corresponding to Table X were included in this pro- 
gram, less than one percent of the terms could be 
identified as completely redundant and removed in 
this manner. Approximately ten percent could be 
identified as having the same entry word, but terms 
following the entry would not match. 

A considerable amount of unique indexing information 
ts provided by each of the services. For this com- 
plex of secondary services, this would seem to be an 
argument in favor of duplicate coverage and processing 
of the same material by more than one service. 



